Automated Measurement of Heavy Equipment Greenhouse Gas Emission: The case of Road/Bridge Construction and Maintenance

Road/bridge construction and maintenance projects are major contributors to greenhouse gas (GHG) emissions such as carbon dioxide (CO2), mainly due to extensive use of heavy-duty diesel construction equipment and large-scale earthworks and earthmoving operations. Heavy equipment is a costly resource and its underutilization could result in significant budget overruns. A practical way to cut emissions is to reduce the time equipment spends doing non-value-added activities and/or idling. Recent research into the monitoring of automated equipment using sensors and Internet-of-Things (IoT) frameworks have leveraged machine learning algorithms to predict the behavior of tracked entities. In this project, end-to-end deep learning models were developed that can learn to accurately classify the activities of construction equipment based on vibration patterns picked up by accelerometers attached to the equipment. Data was collected from two types of real-world construction equipment, both used extensively in road/bridge construction and maintenance projects: excavators and vibratory rollers. The validation accuracies of the developed models were tested of three different deep learning models: a baseline convolutional neural network (CNN); a hybrid convolutional and recurrent long shortterm memory neural network (LSTM); and a temporal convolutional network (TCN). Results indicated that the TCN model had the best performance, the LSTM model had the second-best performance, and the CNN model had the worst performance. The TCN model had over 83% validation accuracy in recognizing activities. Using deep learning methodologies can significantly increase emission estimation accuracy for heavy equipment and help decision-makers to reliably evaluate the environmental impact of heavy civil and infrastructure projects. Reducing the carbon footprint and fuel use of heavy equipment in road/bridge projects have direct and indirect impacts on health and the economy. Public infrastructure projects can leverage the proposed system to reduce the environmental cost of infrastructure project

Data-Driven Simulation Modeling of Construction and Infrastructure Operations Using Process Knowledge Discovery

Within the architecture, engineering, and construction (AEC) domain, simulation modeling is mainly used to facilitate decision-making by enabling the assessment of different operational plans and resource arrangements, that are otherwise difficult (if not impossible), expensive, or time consuming to be evaluated in real world settings. The accuracy of such models directly affects their reliability to serve as a basis for important decisions such as project completion time estimation and resource allocation. Compared to other industries, this is particularly important in construction and infrastructure projects due to the high resource costs and the societal impacts of these projects. Discrete event simulation (DES) is a decision making tool that can benefit the process of design, control, and management of construction operations. Despite recent advancements, most DES models used in construction are created during the early planning and design stage when the lack of factual information from the project prohibits the use of realistic data in simulation modeling. The resulting models, therefore, are often built using rigid (subjective) assumptions and design parameters (e.g. precedence logic, activity durations). In all such cases and in the absence of an inclusive methodology to incorporate real field data as the project evolves, modelers rely on information from previous projects (a.k.a. secondary data), expert judgments, and subjective assumptions to generate simulations to predict future performance. These and similar shortcomings have to a large extent limited the use of traditional DES tools to preliminary studies and long-term planning of construction projects. In the realm of the business process management, process mining as a relatively new research domain seeks to automatically discover a process model by observing activity records and extracting information about processes. The research presented in this Ph.D. Dissertation was in part inspired by the prospect of construction process mining using sensory data collected from field agents. This enabled the extraction of operational knowledge necessary to generate and maintain the fidelity of simulation models. A preliminary study was conducted to demonstrate the feasibility and applicability of data-driven knowledge-based simulation modeling with focus on data collection using wireless sensor network (WSN) and rule-based taxonomy of activities. The resulting knowledge-based simulation models performed very well in properly predicting key performance measures of real construction systems. Next, a pervasive mobile data collection and mining technique was adopted and an activity recognition framework for construction equipment and worker tasks was developed. Data was collected using smartphone accelerometers and gyroscopes from construction entities to generate significant statistical time- and frequency-domain features. The extracted features served as the input of different types of machine learning algorithms that were applied to various construction activities. The trained predictive algorithms were then used to extract activity durations and calculate probability distributions to be fused into corresponding DES models. Results indicated that the generated data-driven knowledge-based simulation models outperform static models created based upon engineering assumptions and estimations with regard to compatibility of performance measure outputs to reality

A Framework For Process Data Collection, Analysis, And Visualization In Construction Projects

Automated data collection, simulation and visualization can substantially enhance the process of designing, analysis, planning, and control of many engineering processes. In particular, managing processes that are dynamic in nature can significantly benefit from such techniques. Construction projects are good examples of such processes where a variety of equipment and resources constantly interact inside an evolving environment. Management of such settings requires a platform capable of providing decision-makers with updated information about the status of project entities and assisting site personnel making critical decisions under uncertainty. To this end, the current practice of using historical data or expert judgments as static inputs to create empirical formulations, bar chart schedules, and simulation networks to study project activities, resource operations, and the environment under which a project is taking place does not seem to offer reliable results. The presented research investigates the requirements and applicability of a data-driven modeling framework capable of collecting and analyzing real time field data from construction equipment. In the developed data collection scheme, a stream of real time data is continuously transferred to a data analysis module to calculate the input parameters required to create dynamic 3D visualizations of ongoing engineering activities, and update the contents of a discrete event simulation (DES) model representing the real engineering process. The generated data-driven simulation model is iv an effective tool for projecting future progress based on existing performance. Ultimately, the developed framework can be used by project decision-makers for shortterm project planning and control since the resulting simulation and visualization are completely based on the latest status of project entities

Robust Activity Recognition for Adaptive Worker-Robot Interaction using Transfer Learning

Human activity recognition (HAR) using machine learning has shown tremendous promise in detecting construction workers' activities. HAR has many applications in human-robot interaction research to enable robots' understanding of human counterparts' activities. However, many existing HAR approaches lack robustness, generalizability, and adaptability. This paper proposes a transfer learning methodology for activity recognition of construction workers that requires orders of magnitude less data and compute time for comparable or better classification accuracy. The developed algorithm transfers features from a model pre-trained by the original authors and fine-tunes them for the downstream task of activity recognition in construction. The model was pre-trained on Kinetics-400, a large-scale video-based human activity recognition dataset with 400 distinct classes. The model was fine-tuned and tested using videos captured from manual material handling (MMH) activities found on YouTube. Results indicate that the fine-tuned model can recognize distinct MMH tasks in a robust and adaptive manner which is crucial for the widespread deployment of collaborative robots in construction.Comment: 2023 ASCE International Conference on Computing in Civil Engineering (I3CE

Expanding Frozen Vision-Language Models without Retraining: Towards Improved Robot Perception

Vision-language models (VLMs) have shown powerful capabilities in visual question answering and reasoning tasks by combining visual representations with the abstract skill set large language models (LLMs) learn during pretraining. Vision, while the most popular modality to augment LLMs with, is only one representation of a scene. In human-robot interaction scenarios, robot perception requires accurate scene understanding by the robot. In this paper, we define and demonstrate a method of aligning the embedding spaces of different modalities (in this case, inertial measurement unit (IMU) data) to the vision embedding space through a combination of supervised and contrastive training, enabling the VLM to understand and reason about these additional modalities without retraining. We opt to give the model IMU embeddings directly over using a separate human activity recognition model that feeds directly into the prompt to allow for any nonlinear interactions between the query, image, and IMU signal that would be lost by mapping the IMU data to a discrete activity label. Further, we demonstrate our methodology's efficacy through experiments involving human activity recognition using IMU data and visual inputs. Our results show that using multiple modalities as input improves the VLM's scene understanding and enhances its overall performance in various tasks, thus paving the way for more versatile and capable language models in multi-modal contexts.Comment: Preprint submitted to Information Fusio

Trust in Construction AI-Powered Collaborative Robots: A Qualitative Empirical Analysis

Construction technology researchers and forward-thinking companies are experimenting with collaborative robots (aka cobots), powered by artificial intelligence (AI), to explore various automation scenarios as part of the digital transformation of the industry. Intelligent cobots are expected to be the dominant type of robots in the future of work in construction. However, the black-box nature of AI-powered cobots and unknown technical and psychological aspects of introducing them to job sites are precursors to trust challenges. By analyzing the results of semi-structured interviews with construction practitioners using grounded theory, this paper investigates the characteristics of trustworthy AI-powered cobots in construction. The study found that while the key trust factors identified in a systematic literature review -- conducted previously by the authors -- resonated with the field experts and end users, other factors such as financial considerations and the uncertainty associated with change were also significant barriers against trusting AI-powered cobots in construction.Comment: 2023 ASCE International Conference on Computing in Civil Engineering (I3CE

Automated Estimation of Construction Equipment Emission Using Inertial Sensors and Machine Learning Models

The construction industry is one of the main producers of greenhouse gasses (GHG). With the looming consequences of climate change, sustainability measures including quantifying the amount of air pollution during a construction project have become an important project objective in the construction industry. A major contributor to air pollution during construction projects is the use of heavy equipment. Therefore, efficient operation and management can substantially reduce a project’s carbon footprint and other environmental harms. Using unintrusive and indirect methods to predict on-road vehicle emissions has been a widely researched topic. Nevertheless, the same is not true in the case of construction equipment. This paper describes the development and deployment of a framework that uses machine learning (ML) methods to predict the level of emissions from heavy construction equipment. Data is collected via an Internet of Things (IoT) approach with accelerometer and gyroscope sensors as data collection nodes. The developed framework was validated using an excavator performing real-world construction work. A portable emission measurement system (PEMS) was used along with the inertial sensors to record the amount of CO, NOX, CO2, SO2, and CH4 pollution emitted by the equipment. Different ML algorithms were developed and compared to identify the best model to predict emission levels from inertial sensors data. The results show that Random Forest with the coefficient of determination (R2) of 0.94, 0.91, and 0.94, and normalized root-mean-square error (NRMSE) of 4.25, 6.42, and 5.17 for CO, NOX, and CO2, respectively, was the best algorithm among different models evaluated in this study

Remote Monitoring Of Dynamic Construction Processes Using Automated Equipment Tracking

Construction resource planning and control is traditionally done using static data and information available from similar projects. However, the uniqueness of and uncertainties involved in each construction project may require that field data from equipment is dynamically collected, analyzed, and integrated into the decision making process in order to achieve the best possible operational plan. The collected data can be used to predict the performance of a construction system based on the latest status of the project, as well as to monitor if all pieces of equipment are operating according to the plan and if any corrective action is needed. This paper presents the results of a remote tracking technique developed to capture field data from construction equipment in real time for short term monitoring and control of construction operations. The developed technique uses a .NET environment thus providing a convenient means for data collection, sorting, filtering, and interpretation. The collected data is time-stamped and thus, can be used to create a real time 3D animation stream of the ongoing operation. This facilitates the communication of project details and can be ultimately used as a verification and validation tool for the underlying simulation model. © 2012 ASCE

Automated Knowledge Discovery And Data-Driven Simulation Model Generation Of Construction Operations

Computer simulation models help construction engineers evaluate different strategies when planning field operations. Construction jobsites are inherently dynamic and unstructured, and thus developing simulation models that properly represent resource operations and interactions requires meticulous input data modeling. Therefore, unlike existing simulation modeling techniques that mainly target long-term planning and close to steady-state scenarios, a realistic construction simulation model reliable enough for short-term planning and control must be built using factual data obtained from ongoing processes of the real system. This paper presents the latest findings of authors\u27 work in designing an integrated data-driven simulation framework that employs a distributed network of sensors to collect multi-modal data from construction equipment activities. Collected data are fused to create metadata structures and data mining methods are then applied to extract key parameters and discover contextual knowledge necessary to create or refine data-driven simulation models that represent the latest conditions on the ground. © 2013 IEEE